OAMIP: Optimizing ANN Architectures Using Mixed-Integer Programming

We introduce a mixed integer program (MIP) for assigning importance scores to each neuron in deep neural network architectures which is guided by the impact of their simultaneous pruning on main learning task network. By carefully devising objective function MIP, we drive solver minimize number critical neurons (i.e., with high score) that need be kept maintaining overall accuracy trained Further, proposed formulation generalizes recently considered lottery ticket optimization identifying multiple "lucky" sub-networks resulting optimized architecture not only performs well single dataset, but also across ones upon retraining weights. Finally, present scalable implementation our method decoupling layers using auxiliary networks. demonstrate ability prune networks marginal loss and generalizability popular datasets architectures.